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A broad consensus for improving vocational education programs has emerged from the 
educational reform movement aimed at raising the performance of U.S. students in 
academic subjects. Pending legislation for the reauthorization of the Carl D. Perkins 
Vocational Education Act (House Bill HR 7 and Senate Bill 1 109) mandates the 
development of performance indicators or standards. In the literature the terms 
educational indicator, quality indicator, outcome indicator, performance standard, and 
performance measure are used interchangeably, and there is general agreement that 
indicators or standards are single or composite statistics that reveal something about 
the performance or health of an educational system (Asche 1990). According to Asche, 
"quality or performance indicators have suddenly become the nation's barometer of 
education wellness" (pp. 3-4). 

A number of options for establishing vocational education performance standards are 
currently under consideration. This ERIC Digest examines some vocational education 
experiences with outcome measures, describes proposed approaches, and enumerates 
potential issues and challenges in establishing performance standards for vocational 
education. 

VOCATIONAL EDUCATION OUTCOME 
MEASURES 

During the last decade, state and local programs have developed experience with 
defining and measuring performance-oriented outcomes of vocational education. Three 
of the most frequently used outcome measures are labor market, learning, and access. 
Although each of these has its strengths and weaknesses, they reflect the broadly 
accepted definition of the principal objectives of vocational training-the preparation of 
individuals for productive and gainful employment-as well as a primary policy concern 
with program access (Hoachlander, Choy, and Brown 1989; Office of Technology 
Assessment 1989). 

LABOR MARKET OUTCOMES 



Indicators of labor market performance of vocational graduates-the traditional 
standards by which the effectiveness of vocational education and employment training 
have been measured-include job placement, earnings, and duration of employment 
and unemployment. Although economic indicators have been widely used as a measure 
of vocational education's effectiveness, there are problems with the traditional methods 
of collecting the data. Information provided by program graduates has questionable 
validity, and there is also potential for bias in data provided by school personnel. Using 
state wage records as a means of collecting more accurate information has been 
proposed as a way to address some of these concerns (Hoachlander, Choy, and Brown 
1989; OTA 1989). 

Even if the labor market outcome information could be collected in a valid, unbiased, 
and accurate manner, many vocational educators would object to such indicators being 
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the sole measure of program effectiveness on the following grounds: (1) adopting 
placement as the primary criterion ignores the multiple goals of vocational education; (2) 
a large number of economic and personal factors beyond the control of the vocational 
education system determine the employment of students; (3) a narrow focus on 
placement encourages programs to admit only those who can be placed and to 
concentrate on coaching in job placement and interview skills at the expense of 
vocational skills; and (4) placement rates and other economic indicators measure the 
gross effect of participation (total place, total earnings) rather than the net effect (the 
difference between labor market outcomes that occurred when students participated in 
vocational education versus what would have occurred had these programs not existed) 
(ibid.). 

LEARNING OUTCOMES 

Learning outcomes--what individuals learn in school-are at least as important an 
indicator of program quality as labor market outcomes. In addition, vocational educators 
have much more control over what and how much students learn than they do over 
what happens to them in the labor market. 

Although there are several ways to measure learning outcomes, the most common 
method employed in vocational education is occupational competency testing designed 
to assess mastery of skills (tasks) and knowledge found in specific jobs. Even though 
competency testing has not played a role in federal vocational education policy, states 
have been quite active in developing and using competency tests (Hoachlander, Choy, 
and Brown 1989). 

Occupational competency tests provide important indications of program effectiveness, 
but many vocational educators do not think they should be used as the sole basis for 
performance standards for the following reasons: test scores often reflect economic and 
social factors in addition to abilities developed through education and training; schools 
may coach students on test-taking strategies and on specific test items at the expense 
of teaching the skills and knowledge purportedly measured by the test; tests do not 
necessarily indicate how a person would perform at work but instead measure the upper 
limit of what an individual can do; and at a time when there is a growing consensus that 
more broadly applicable generic skills are needed in the workplace, use of competency 
tests could encourage emphasis on highly specialized skills (OTA 1989). 

ACCESS OUTCOMES 

Because federal policy has emphasized access to vocational programs for women, 
minorities, and students with special needs, outcomes related to access are also under 
consideration as performance indicators. Traditional efforts to monitor access outcomes 
in vocational education have focused on monitoring the numbers of different types of 
students enrolled in particular programs relative to their numbers in the larger 



ED318914 1990-00-00 Vocational Education Performance Standards. ERIC Digest No. 
96. 



Page 3 of 6 



www , eric , ed , gov 



ERIC Custom Transformations Team 



population. However, these efforts have suffered from two major flaws (Hoachlander, 
Choy, and Brown 1989). 

First, when enrollments are used to monitor access, attention is deflected from the 
primary policy concern that these students acquire the skills necessary to compete 
effectively in the labor market. Second, monitoring access has traditionally relied on 
program-based data collection rather than course-based efforts, and the result has been 
a "series of inaccurate, inconsistent, and generally chaotic attempts to arbitrarily assign 
students to vocational education programs" (ibid., p. 44). To correct these problems, 
Hoachlander, Choy, and Brown (1989) suggest tying access into performance-based 
policies as well as relying on periodic collection of student transcripts, which would 
permit longitudinal analysis, to monitor access. 

SUGGESTED APPROACHES 



None of the outcome measures currently in use appears to be sufficient for judging the 
quality of vocational education programs when used singly. However, some 
combination of labor market outcomes, learner outcomes, and access outcomes seems 
to hold promise for developing measurable standards of performance. States that either 
have implemented or have under development systems to measure performance vary in 
their approaches, especially in terms of the level of specificity of the indicators, the level 
at which the data are collected and used, and the primary purpose of the system (Asche 
1990). 

Illinois is currently pilot testing a technically advanced system that uses quality 
indicators to link planning, evaluation, and program improvement through data that are 
collected and analyzed at either the local or regional level. The system uses six 
indicators for measuring outcomes: placement and continuing education, enrollment, 
employer satisfaction, student satisfaction, employability skills attainment, and cost. 
Although the primary focus of the Illinois system is program improvement, it appears to 
satisfy most of the requirements in the pending federal legislation (Asche 1990). 

Copa and Scholl (1 983) report on efforts to verify and develop a set of indicators for use 
in measuring the outcomes of vocational education in Minnesota. A method of 
verification was used that took into consideration current social and economic issues, 
vocational education as a part of education more broadly conceived, and critical 
questions useful in examining any potential indicator. Use of this verification process 
resulted in the selection of four vocational education indicators that appeared to have 
adequate data available to support continued development: number of graduates 
employed, number of graduates employed in occupations related to program, 
employer's satisfaction with the quality of the graduate's work, and program cost. 
Although these indicators are heavily oriented toward labor market outcomes, the 
process described is useful for those developing systems of measuring vocational 
outcomes. 
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ISSUES AND CHALLENGES 

Vocational education is a multifaceted system with a diverse clientele and multiple 
goals, and it exists in a complex policy environment. Developing and implementing a 
system of performance standards for vocational education requires making demanding 
decisions on performance assessment, accountability, and actions. 
Apling (1989) describes a set of potential problems that any performance standards 
system for vocational education needs to address: the impact of performance standards 
on those whom the program serves, with the danger that individuals most needing 
services will be the least likely to be served; the influence of performance standards on 
the types of training provided, with a danger that effective--but long-term and 
expensive-services will be discouraged in favor of short-term and inexpensive 
approaches; the difficulty in meeting multiple standards, some of which may not be 
compatible; the problem of adjusting standards for programs in different labor markets 
or serving different types of clients; and the difficulty of setting minimum standards (p. 
18). 

Apling also suggests that such questions as what performance is assessed, whose 
performance is assessed, who uses performance information and how do they use it, 
and what results from meeting, not meeting, or exceeding standards raise issues that 
must be resolved in order to implement a performance standard system for vocational 
education. 

Potentially, any system of performance standards developed for use in vocational 
education can have both positive and egative effects. Asche (1990) cautions vocational 
educators to be aware that indicators tend to measure quantity, not quality; are 
designed to serve policymakers, not educators; aim for parsimony, not complexity; tend 
to reflect those things that can be easily measured; and unless there is reasonable 
agreement on what educational goals should be, may shape the curriculum (p. 5). On 
the other hand, Asche also points out that a performance indicator system can have 
positive aspects: locally developed indicators can provide opportunities for 
school-based improvement and the development of shared goals and values, indicators 
can be useful in monitoring policies and practice and improving schools, and indicators 
offer an opportunity for vocational education to be included in educational reforms (p. 6). 
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